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1.  Introduction 


Autonomous  agents  have  been  increasingly  used  for  military  operations  (e.g.,  casualty 
extraction,  explosive  detection  and  disposals,  reconnaissance  and  surveillance,  supply 
transportation,  building  clearing,  and  firefighting),  search  and  rescue,  transportation  safety,  space 
exploration,  farming,  and  many  other  purposes  yet  to  emerge  (Bames  and  Evans,  2010; 
Greenemeier,  2010;  Jones  and  Schmidlin,  2011;  “Morals  and  the  Machine,”  2012;  Murphy  and 
Burke,  2010;  Osbom,  2011;  Purdy,  2008;  Singer,  2010).  As  these  agents  become  more 
sophisticated  and  independent,  it  is  critical  for  their  human  counterparts  to  understand  their 
behaviors,  the  reasoning  process  behind  those  behaviors,  and  the  expected  outcomes  to  properly 
calibrate  their  trust  in  the  systems  and  make  appropriate  decisions  (de  Visser  et  al.,  2014;  Lee 
and  See,  2004).  Indeed,  past  studies  have  shown  that  human  operators  sometimes  question  the 
accuracy  and  effectiveness  of  agents’  actions  due  to  the  operators’  difficulties  understanding  the 
state/status  of  the  agent  (Bitan  and  Meyer,  2007;  Seppelt  and  Lee,  2007;  Stanton  et  al.,  2007)  and 
the  rationales  behind  the  behaviors  (Linegang  et  al.,  2006).  These  limitations  can  be  substantial 
impediments  to  overall  system  and  task  performance. 

Lee  and  See  (2004)  identified  three  antecedents  for  trust  development  in  the  context  of  human- 
agent  interaction:  purpose,  process,  and  performance.  The  purpose  factor  deals  with  the  degree  to 
which  the  agent-driven  automation  is  being  used  according  to  the  designer’s  intent.  For  example, 
an  operator  should  understand  his/her  unmanned  vehicle’s  current  purpose,  such  as  threat 
detection  along  a  route.  The  process  factor  deals  with  the  question  of  whether  the  algorithm  of 
the  automated  system  is  appropriate  for  a  given  task.  Lor  example,  in  an  unmanned  vehicle,  the 
underlying  algorithm  could  be  governing  whether  the  system  only  monitors  a  given  area  or 
actually  identifies  threats  within  a  certain  distance,  and  the  operator  should  understand  the  uses 
and  limitations  of  the  agent  based  on  the  integrity  of  its  process.  The  performance  factor  deals 
with  system  reliability,  predictability,  and  capability.  Lor  example,  the  operator  should  know  that 
the  unmanned  vehicle  tends  to  be  unreliable  in  a  certain  terrain  environment  but  can  effectively 
communicate  with  the  command  center  and  accurately  mark  threats  with  a  laser  pointer.  Lee 
(2012)  proposed  that  to  increase  system  transparency  to  the  operator,  the  system’s  3Ps  (purpose, 
process,  and  performance),  as  well  as  the  history  of  the  3Ps,  should  be  visible  to  the  operator. 
However,  the  presentation  should  be  in  a  simplified  form  (e.g.,  integrated  graphical  displays)  so 
the  operator  is  not  overwhelmed  by  the  additional  information  he/she  needs  to  process  (Cook  and 
Smallman,  2008;  Neyedli  et  al.,  2011;  Tufte,  2001). 
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2.  SAT  Model 


We  propose  a  model  of  agent  transparency  to  support  operator  situation  awareness  (SA)  of  the 
mission  environment  involving  the  agent,  the  SA-based  Agent  Transparency  (SAT)  (figure  1), 
based  on  the  theory  of  SA  (Endsley,  1995),  the  Beliefs,  Desires,  Intentions  (BDI)  Agent 
Framework  (Rao  and  Georgeff,  1995),  Lee’s  3Ps  described  previously  (Lee,  2012),  and  other 
relevant  previous  work  (Chen  and  Barnes,  2012a,  2012b;  Chen  and  Barnes,  2014;  Cring  and 
Lenfestey,  2009;  Lyons  and  Havig,  in  press).  According  to  Endsley  (1995),  SA  is  “a  state  of 
knowledge”  of  “the  state  of  a  dynamic  environment”  (p.  36).  It  has  three  levels,  to  include  (1) 
perception  of  basic  elements,  (2)  comprehension  of  those  elements’  meaning  when  taken  as  a 
whole,  and  (3)  the  ability  to  project  the  system’s  status  in  the  near-term  based  on  that 
comprehension.  We  have  defined  agent  transparency  as  the  descriptive  quality  of  an  interface 
pertaining  to  its  abilities  to  afford  an  operator’s  comprehension  about  an  intelligent  agent’s 
intent,  performance,  future  plans,  and  reasoning  process.  From  Endsley’s  model  we  borrowed  the 
concept  of  identifying  features  of  the  environment  necessary  for  global  SA.  Endsley’s  model 
refers  to  cognitive  requirements  for  different  levels  of  global  SA  whereas  the  SAT  model  refers 
to  transparency  requirements  to  understand  the  intelligent  agent’s  task  parameters,  logic,  and 
predicted  outcomes. 

•  To  support  operator’s  Level  1  SA  (What’s  going  on  and  what  is  the  agent 
trying  to  achieve?) 

•  Purpose 

•  Desire  (Goal  selection) 

•  Process 

•  Intentions  (Planning/Execution) 

•  Progress 

•  Performance 

•  To  support  operator ’s  Level  2  SA  (Why  does  the  agent  do  it?) 

•  Reasoning  process  (Belief  (Purpose) 

•  Environmental  &  other  constraints  (Belief 

•  To  support  operator’s  Level  3  SA  (What  should  the  operator  expect  to 
happen?) 

•  Projection  to  Future/End  State 

•  Potential  limitations 

•  Likelihood  of  error 

•  History  ofperformance 


Figure  1.  SA-based  Agent  Transparency  model. 

At  the  first  level  of  the  SAT  model,  the  operator  is  provided  with  the  basic  information  about  the 
agent’s  current  state  and  goals,  intentions,  and  proposed  actions.  At  the  second  level,  the  operator 
is  provided  information  about  the  agent’s  reasoning  process  behind  those  actions  and  the 
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constraints/affordances  that  the  agent  considers  when  planning  those  actions.  At  the  third  level, 
the  operator  is  provided  with  information  regarding  the  agent’s  projection  of  the  future  state, 
such  as  predicted  consequences,  likelihood  of  success/failure,  and  any  uncertainty  associated 
with  the  aforementioned  projections. 

Incorporating  all  three  levels  should  allow  an  operator  to  gain  understanding  of  an  agent’s 
reasoning  process  behind  its  actions  and  help  the  operator  make  informed  decisions  as  to  whether 
he  or  she  should  intervene.  Additionally,  the  SAT  model  differs  from  Endsley’s  (1995)  SA 
model  in  that  not  all  three  levels  may  be  absolutely  necessary  to  achieving  transparency — they 
are  simply  different  aspects.  In  Endsley’s  model,  SA  is  the  cumulative  result  of  the  levels  while 
transparency  may  not  be.  For  example,  in  a  time-sensitive  situation,  the  operator  may  only  need 
to  know  the  agent’s  proposed  actions  (level  1)  and  the  projected  outcome  (level  3)  to  make  a 
sufficiently  informed  decision.  We  will  now  discuss  operator  trust  and  workload  in  autonomous 
systems  through  the  lens  of  the  SAT  model. 


3.  Implications  for  Operator  Trust  and  Workload 


Automation  reliability  strongly  influences  an  operator’s  attitude  toward  the  automation,  which  in 
turn  affects  their  level  of  trust  (Hancock  et  al.,  2011;  Lee  and  See,  2004;  Parasuraman  and  Riley, 
1997;  Schaefer  et  al.,  in  press).  This  then  affects  whether  the  operator  will  decide  to  use  the 
automation,  which  is  also  known  as  the  automation  usage  decision  (AUD)  (Beck,  et  al.,  2007). 
Inappropriate  levels  of  trust  relative  to  the  automation’s  reliability  rate  result  in  improper 
automation  use.  According  to  Parasuraman  and  Riley  (1997),  overtrusting  automation  results  in 
automation  misuse,  in  which  an  operator  may  become  complacent  and  overrely  on  the 
automation.  Automation  overreliance  is  damaging  to  SA  and  results  in  accidents  as  operators 
may  fail  to  monitor  the  system  properly.  This  leads  both  to  errors  of  omission  (failing  to  notice  a 
problem  because  the  automation  did  not  alert  the  operator)  and  commission  (following  the 
automation  when  there  are  indicators  that  it  is  incorrect).  Lee  and  See  (2004)  refer  to  this  as 
overtrust.  Automation  disuse  stems  from  a  lack  of  trust  in  the  automation,  most  likely  due  to  a 
high  false  alarm  rate  (Parasuraman  and  Riley,  1997).  Proper  automation  use,  then,  is  made 
possible  as  a  result  of  awareness  and  understanding  of  the  automation’s  reliability  and  other 
relevant  factors  (i.e.,  transparency)  resulting  in  the  development  of  an  appropriate  level  of  trust, 
which  Lee  and  See  (2004)  refer  to  as  calibrated  trust.  Including  the  appropriate  level  of 
information  in  the  interface  should  increase  agent  transparency.  This  promotes  the  development 
of  proper  trust  calibration,  which  results  in  an  appropriate  automation  usage  decision.  Ultimately, 
this  improves  operator  performance.  This  relationship  illustrated  in  figure  2. 
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Figure  2.  Relationship  between  transparency  and  performance. 

The  relationship  between  the  automation’s  accuracy  and  the  AUD  can  be  analyzed  in  terms  of 
Signal  Detection  Theory  (SDT)  (Green  and  Swets,  1966).  A  signal  can  be  defined  as  every  time 
the  automation  makes  or  executes  its  own  suggestion  and  is  accurate,  and  the  operator  either 
accepts  that  decision  or  does  not  opt  to  override  the  automation’s  execution  strategy.  When  the 
automation  is  making  a  correct  suggestion,  accepting  the  automation’s  suggestion  is  akin  to  a  hit. 
When  the  automation  is  wrong  but  the  operator  accepts  the  suggestion  anyway,  it  is  a  false 
alarm.  Failing  to  use  the  automation  when  it  is  correct  would  be  a  miss  while  not  using  the 
automation  when  it  was  wrong  would  be  a  correct  rejection. 

Another  way  to  evaluate  the  AUD  is  to  consider  reliance  and  compliance.  Meyer  et  al.  (in  press) 
define  reliance  and  compliance  as  types  of  trust  in  terms  of  the  response  to  a  binary  alert. 
Compliance  is  “the  degree  to  which  the  binary  cue,  when  it  is  present,  causes  the  operator  to  act 
in  accordance  to  the  cue”  (Meyer,  et  al.,  p.  1),  while  reliance  is  “the  degree  to  which  operators 
dare  to  avoid  taking  precautions  when  the  binary  indicator  does  not  point  to  a  signal.” 
Essentially,  compliance  means  taking  action  based  on  a  cue  while  reliance  is  not  doing  anything 
in  the  absence  of  an  alert. 

In  the  AUD  scenario,  reliance  occurs  when  the  operator  trusts  the  automation  to  alert  them  if 
there  is  a  problem;  otherwise  they  will  not  take  any  independent  actions  or  override  the 
automation’s  suggestions  or  actions.  Compliance  occurs  when  a  decision  must  be  made  and  the 
operator  chooses  to  go  along  with  the  automation.  We  would  like  to  think  of  reliance  and 
compliance  in  a  somewhat  novel  way,  especially  since  not  all  instances  of  whether  an  operator 
will  rely  on  an  agent  will  be  based  on  alerts.  Reliance  can  be  considered  as  the  number  of 
sources  an  operator  uses  to  make  a  decision — to  include  the  agent — while  compliance  is  whether 
the  operator  chooses  to  adhere  to  the  agent’s  suggestion  or  proposed  plans.  For  example,  if  an 
operator  only  uses  the  agent  to  make  decisions,  rather  than  consulting  outside  sources  such  as 
intelligence  reports,  they  have  high  reliance,  but  if  they  do  not  tend  to  carry  out  the  agent’s 
suggested  course  of  action,  they  have  low  compliance.  Conversely,  when  an  operator  uses 
multiple  sources  to  make  decisions  it  is  considered  low  reliance,  but  if  he/she  complies  with  the 
agent’s  input  without  much  modification  when  it  is  adopted  it  is  considered  high  compliance. 

See  table  1  for  a  representation  of  AUD  in  terms  of  usage,  trust,  SDT,  and  compliance.  Only 
compliance  is  described  because,  in  this  context,  automation  usage  implies  relying  on  the 
automation  to  provide  recommendations  and  then  choosing  to  comply. 
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Table  1.  Automation  usage  decision  matrix. 


State  of 
Automation 

Operator  AUD 

Usage3 

Trustb 

SDTC 

Compliance11 

Accurate 

Accepts 

Proper  use 

Calibrated  trust 

Hit 

Appropriate  compliance 

Accurate 

Overrides, 

incorrectly 

adjusts 

Disuse 

Distrust 

Miss 

Inappropriate 

noncompliance 

Inaccurate 

Overrides, 
correctly  adjusts 

Proper  disuse 

Calibrated  trust 

Correct 

rejection 

Appropriate 

noncompliance 

Inaccurate 

Accepts 

Misuse 

Overtrust 

False  alarm 

Inappropriate  compliance 

Inaccurate 

Overrides, 

incorrectly 

adjusts 

Error 

Error 

Error 

Erroneous  noncompliance 

“Parasuraman  and  Riley,  1997;  bLee  and  See,  2004;  LGreen  and  Swets,  1966;  dadapted  from  Meyer  et  al.,  in  press. 


The  goal  of  using  transparent  interfaces  is  to  achieve  both  high  rates  of  proper  use  when 
automation  is  accurate  and  high  rates  of  proper  disuse  when  it  is  not.  Rates  of  disuse,  misuse,  and 
errors  should  be  much  lower  if  the  interface  is  transparent.  Knowledge  of  the  system’s 
unreliability  will  shape  an  operators  trust  in  the  agent  (Lee  and  See,  2004).  Overtrusting  a  system 
will  increase  the  likelihood  of  automation  misuse,  and  undertrusting  a  system  will  increase  the 
likelihood  of  automation  disuse  (Parasuraman  and  Riley,  1997;  Lee  and  See,  2004).  It  is 
important  to  be  clear:  it  is  not  expected  that  trust  will  increase  with  transparency.  For  example, 
revealing  more  information  should  make  a  person  trust  the  interface  less  if  the  operator 
understands  it  to  be  not  reliable.  Therefore,  it  is  likely  that  high  levels  of  agent  transparency  will 
result  in  appropriate  trust  calibration  (e.g.,  low  levels  of  trust  when  the  automation  is  unreliable, 
high  levels  when  it  is  perfectly  reliable,  and  moderate  levels  when  it  is  somewhat  unreliable), 
which  will  then  result  in  appropriate  AUDs.  We  will  now  address  how  to  support  agent 
transparency  through  various  user  interface  design. 


4.  Supporting  Transparency  Through  User  Interface  Design 


Automation  transparency  has  been  characterized  by  observability,  directability,  adaptability,  and 
broadening  (DePass  et  al.,  2011;  Scott  et  al.,  2009;  Truxler  et  al.,  2012).  Observability  refers  to  a 
shared  frame  of  reference  for  the  automation  and  operator.  Directability  refers  to  allowing  the 
operators  to  modify  assumptions  and  guide  problem  solving  via  multiple  mechanisms. 
Adaptability  refers  to  supporting  incremental  commitment  of  resources  and  reserving  degrees  of 
freedom  to  accommodate  changes.  Broadening  refers  to  enabling  operators  to  compare  multiple 
solutions  in  a  “what  if’  mode.  These  four  elements  should  be  included  in  the  user  interface  to 
promote  transparency.  Increasing  the  level  of  transparency,  however,  requires  including  more 
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information  in  an  interface.  This  has  implications  for  operator  workload.  As  the  level  of 
information  increases,  operator  workload  should  also  increase  as  the  operator  will  have  to  read 
and  process  more  visual  information  than  if  less  had  been  provided.  This  can  be  mitigated,  in 
part,  by  offloading  some  complex  computation  by  calculating  projections  of  success — a  part  of 
level  3  SAT — for  the  operator.  Still,  Lee  (2012)  proposed  that  integrated  displays  of 
transparency  information  may  further  reduce  workload. 

4.1  Information  Visualization 

Transparency  information  in  the  form  of  both  text  and  images  can  be  integrated  into  an  interface 
through  what  is  known  as  information  visualization,  which  has  been  defined  as  “the  use  of 
computer-supported,  interactive,  visual  representations  of  abstract  data  in  order  to  amplify 
cognition”  (Card  et  al.,  1999,  p.  7),  and  its  goal  is  to  support  user  understanding  of  the  presented 
information  (Heer  et  al.,  2010).  Visualizations  may  be  interpreted  from  a  top-down  perspective, 
also  known  as  conceptually  driven  processing.  An  individual  applies  their  previous  knowledge, 
experiences,  and  motivations  to  how  they  interpret  a  given  information  visualization.  Previous 
research  in  understanding  the  interpretation  of  instructions  and  symbols  indicates  that  individuals 
who  are  familiar  with  the  task  of  interpreting  designs  perform  much  more  efficiently  than  those 
who  are  unfamiliar  with  such  tasks  (Tang  et  al.,  2008).  This  is  also  found  when  examining  task 
performance  between  novice  users  and  experts  (Charness,  1981).  Charness  noticed  that  the 
difference  between  novice  chess  players  and  experts  was  that  experts  were  able  to  remember 
different  search  strategies  and  patterns,  thereby  relying  more  on  predefined  schema.  A  schema  is 
an  understanding  of  the  general  aspects  of  a  given  stimulus  or  situation  (D ’Andrade,  1995; 
Kellogg,  1995).  It  is  through  practice  and  experience  that  these  schemata  are  developed,  which 
results  in  the  development  of  a  situational  mental  model.  These  mental  models  help  us  to 
describe  and  predict  the  world  around  us  (Rouse  and  Morris,  1986).  Relying  on  preexisting 
schemata  helps  operators  develop  the  mental  model  for  the  new  data  represented  in  the 
information  visualization. 

The  Ecological  Interface  Design  (EID)  strategy  minimizes  workload  by  relying  on  top-down 
processing  and  existing  schemata.  EID  is  used  to  build  interpretation  affordances  into  interfaces, 
eschewing  convention  in  favor  of  intuition  so  that  information  is  quickly  understood.  It  achieves 
this  by  using  information  visualization  to  “map  such  abstract  properties — and  their  tightly 
coupled  relationships  across  system  components,  processes,  and  operational  goals — to  readily 
perceived  visual  characteristics  of  interface  display  elements  (e.g.,  the  thickness,  angular 
orientation,  or  color  of  a  line;  the  size  or  transparency  of  an  icon)”  (Kilgore  and  Voshell,  in 
press,  p.  3).  The  effectiveness  of  EIDs  for  human-agent  interaction  has  been  investigated  in 
several  studies  (Cummings  and  Bruni,  2010;  Furukawa  et  al.,  2004;  Furukawa  and  Parasuraman, 
2003;  Kirlik,  1995;  Seppelt  and  Lee,  2007;  Vicente  and  Rasmussen,  1990;  Vicente  and 
Rasmussen,  1992;  Vicente,  2002;  Woods,  1991).  In  these  studies,  EIDs  have  been  used  to 
graphically  portray  the  capabilities  and  limitations  of  the  agent,  intent  of  the  agent,  and  quality  of 
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plan  revisions  proposed  by  the  agent.  These  studies  consistently  found  that  EIDs  were  effective 
in  supporting  operators’  understanding  of  agents’  behavior  as  well  as  predictions  of  agents’ 
future  behaviors  and,  thereby,  promoting  proper  operator  trust  calibration. 

Kosara  et  al.  (2003)  noted  that  one  of  the  distinguishing  points  of  information  visualization  is 
that  it  can  convey  the  meaning  of  data  without  necessarily  relying  on  an  existing  mental  model. 
At  the  very  fundamental,  bottom- up  level,  Bertin  (1983)  attempted  to  create  a  classification 
structure  for  the  various  symbolic  elements  that  result  in  responses  from  the  visual  system,  which 
he  denoted  as  marks.  Healey  et  al.  (1996)  described  mark  combinations  as  preattentive  cues, 
which  are  those  items  that  can  be  cognitively  processed  prior  to  attending  to  a  specific  stimulus. 
A  cue  has  characteristics  that  allow  it  to  stand  apart  from  the  area  around  it  (Heer  et  al.,  2010; 
Treisman,  1985).  Examples  of  preattentive  cues  that  are  relevant  to  this  work  are:  number, 
length,  closure,  hue,  size,  and  curvature.  Kossyln  (1989)  developed  recommendations  for  how  to 
turn  marks  into  chart  data.  Marks  need  to  be  individualized,  unique,  and  distinguishable.  There 
also  should  be  a  direct  mapping  between  a  mark  and  an  interpretation  of  a  piece  of  data.  Even 
high-density  data  in  combination  with  simple,  clear  graphics  provide  complete  and 
understandable  information  (Guttormsen,  2012). 

Top-down  mental  models  combined  with  the  preattentive  cues  from  bottom-up  processing  shape 
how  an  individual  assesses  and  understands  his/her  situation.  In  an  automated  agent’s  interface, 
well-designed  information  visualizations  promote  transparency  and  result  in  properly  calibrated 
trust.  Both  top-down  and  bottom-up  processing  can  help  guide  designers  as  to  how  particular 
visualizations  should  be  designed,  but  the  SAT  model  can  be  used  to  determine  specifically  what 
types  of  visualizations  should  be  incorporated  in  an  agent’s  interface  to  make  it  more  transparent. 
The  SAT  model  delineates  several  specific  pieces  of  information  that  can  be  visualized  to 
support  each  level  of  understanding.  In  the  following,  specific  types  of  information  that  support 
each  level  of  SAT  are  listed,  and  for  each  type  of  information,  the  relevant  information 
visualization  literature  has  been  reviewed  and  examples  provided. 

4.2  Displaying  Uncertainty  Information 

Prediction  is  an  important  criterion  for  understanding  plans  developed  by  intelligent  agents.  A 
disconnect  between  the  intent  of  the  operator  and  the  predicted  outcome  should  signal  the 
operator  to  consider  alternate  plans.  To  further  complicate  matters,  in  complex  environments, 
outcomes  are  rarely  deterministic,  and  operators  must  assess  the  relative  uncertainty  of  options  as 
well  as  their  expected  outcomes.  Decisionmaking  in  uncertainty  is  difficult,  and  there  is  strong 
evidence  that  humans  use  a  variety  of  heuristics  and  rules  of  thumb  to  rapidly  assess  probabilities 
in  real  world  situations  that  may  lead  to  satisfactory  solutions  or  in  other  cases  to  erroneous 
conclusions  (Kahneman  and  Klein,  2009;  Wickens  and  Hollands,  2000).  A  famous  example  is 
the  so-called  Monty  Hall  (of  “Let’s  Make  a  Deal”  television  show)  problem:  the  contestant  picks 
which  of  three  doors  (1/3  probability  each)  has  the  prize  behind  it.  When  the  contestant  chooses 
a  door,  Monty  (not  so  helpfully)  opens  one  of  the  remaining  two  doors  that  does  not  have  the 
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prize  behind  it  and  gives  the  contestant  a  chance  to  switch.  Most  often,  contestants  stay  with  the 
original  choice  assuming  that  the  two  remaining  unopened  doors  are  equally  probable.  In  fact, 
contestants  are  better  served  by  switching  doors  because  the  remaining  unchosen  door  actually 
has  a  two-thirds  probability  of  success — Monty  always  opened  the  non-prize  door,  thus  there  is 
still  a  two-thirds  chance  for  the  remaining  unchosen  door  to  have  the  prize  behind  it.  Johnson- 
Laird  et  al.  (1999)  indicated  that  the  Monty  Hall  example  is  not  an  isolated  phenomenon;  there 
are  a  number  of  “probability  illusions”  that  deceive  even  well-educated  participants.  In  general, 
they  conclude  that  humans  use  an  incorrect  mental  model  of  the  processes  generating  the 
probabilities,  tending  to  give  less  weight  to  possible  negative  outcomes  (Johnson-Laird  et  al., 
2004).  The  illusions  are  strong,  and  it  is  difficult  to  convince  participants  that  the  normative 
solutions  are  correct.  Probability  estimates  are  also  influenced  by  stereotypes  and  human  biases 
(Tversky  and  Kahneman,  1974),  inability  to  enumerate  possible  outcomes  (Rottenstreich  and 
Tverky,  1997),  overconfidence  (Fischhoff  et  al.,  1997),  context  of  the  information  (Moldoveanu 
and  Langer,  2002),  and  emotional  responses  to  possible  losses  (Barnes  et  al.,  201 1). 

Kahneman  and  Tversky’s  (1979)  research  on  Prospect  Theory  demonstrated  that  supplying 
humans  with  veridical  probabilities  does  not  overcome  cognitive  biases.  Prospect  Theory  posits 
two  important  human  errors:  (1)  extreme  probabilities  in  either  direction  are  misinterpreted  (see, 
also,  Gonzalez  and  Wu,  1999)  and  (2)  humans  interpret  probabilities  in  terms  of  their  loss/gain 
values.  The  same  information  expressed  in  terms  of  a  loss  led  to  different  decisions  than  its 
complement,  probability  of  success.  Decisions  framed  in  terms  of  loses  resulted  in  risk-seeking 
behaviors;  that  is,  humans  avoid  taking  a  sure  loss  in  the  present  rather  than  opting  for  a  possible 
more  severe  loss  in  the  future.  Given  the  same  decision  framed  in  terms  of  success,  humans  are 
risk-averse  and  more  likely  to  choose  the  sure  thing  rather  than  a  riskier  option  with  a  higher 
payoff  in  the  future.  A  current  example  is  climate  change;  even  with  overwhelming  evidence  of 
change,  humans  are  reluctant  to  make  necessary  reforms  now  (sure  loss  in  terms  of  money)  even 
though  the  consequences  of  waiting  could  be  catastrophic  (future  loss). 

Recent  research  indicates  that  loss  aversion  is  a  more  general  phenomenon  than  Prospect  Theory 
because  (1)  loss  aversion  affects  nonrisky  decisions  as  well  as  risky  ones,  (2)  neurophysiological 
recordings  of  loss  information  in  the  amygdala  suggest  emotional  reactions  to  loss,  and  (3)  there 
is  evidence  that  processing  is  less  effective  for  loss  information  (Barnes  et  al.,  2011;  Nygren, 
1997;  Shafir  and  Tversky,  1995).  The  misinterpretation  of  extreme  probabilities  is  exacerbated 
by  the  scales  used.  For  example,  an  increase  of  0.95  to  0.99  appears  slight  but  an  increase  of  19:1 
odds  to  99: 1  odds  indicates  a  five-fold  increase  even  though  they  represent  the  exact  same 
likelihood.  There  is  evidence  that  presenting  information  in  terms  of  frequencies  instead  of 
numerical  probabilities  improves  Bayesian  predictions  (Gigerenzer  and  Hoffrage,  1995); 
however,  other  researchers  have  found  that  the  cognitive  illusions  previously  mentioned  are  not 
improved  by  frequency  interpretations  of  uncertainty  (Griffin  and  Buehler,  1999). 
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A  number  of  experimenters  found  evidence  that  spatial  representations  and  graphic  displays  of 
uncertainty  improved  decision  accuracy,  but  not  in  all  cases,  and  graphic  displays  did  not 
ameliorate  loss  aversion  affects  (Barnes  et  al.,  2011;  Kirschenbaum  and  Arruda,  1994). 

Interestingly,  Israeli  and  American  researchers  found  instances  when  probability  assessment 
degraded  performance  compared  with  verbal  cues  for  making  normative  choices  related  to  bets 
about  the  outcome  of  basketball  games  (Erev  et  al.,  1993).  However,  in  a  different  context, 

Bass  et  al.  (2013)  found  that  graphical  uncertainty  information  was  an  effective  addition  to 
aircraft  status  and  predictive  information  for  automated  air  traffic  control.  The  additional 
information  was  presented  in  terms  of  noise  information  (mean  and  standard  deviation  as 
represented  by  color-coding  and  symbology)  around  key  automation  strategies  (speed  and 
heading  as  represented  by  projected  path  overlays)  indicating  the  inherent  uncertainty  involved 
as  limits.  This  was  more  effective  than  just  presenting  the  automated  system’s  scalar  probability 
estimate  or  presenting  the  estimate  and  the  logic  underlying  the  estimate.  However,  Bass  et  al. 
(2013)  also  found  that  while  providing  information  about  environmental  uncertainty  through  the 
use  of  colors  and  symbology  is  beneficial,  adding  projected  path  information  to  the  display  does 
not  significantly  increase  accuracy.  In  this  case,  more  information  was  not  necessarily  better. 
Although  the  literature  is  anything  but  clear  as  to  the  best  way  to  represents  uncertainty,  a 
number  of  general  principles  were  suggested: 

•  Numerical  probabilities  are  confusing  and  alternative  representations  of  uncertainty  such  as 
confidence  limits  or  ranges  should  be  considered  (Bass  et  al.,  2013;  Erev  et  al.,  1993; 
Gigerenzer  and  Hoffrage,  1995). 

•  Graphical  representations  are  easier  for  humans  to  process  than  numerical  representations 
in  order  to  gauge  relative  uncertainty  values  (Kirschenbaum  and  Arruda,  1994;  Wickens 
and  Hollands,  2000). 

•  In  military  situations,  outcomes  should  be  presented  in  terms  of  mission  success  instead  of 
possibility  of  mission  failure  (Barnes  et  al.,  2011;  Nygren,  1997;  Shafir  and  Tversky, 

1995). 

•  Especially  for  extreme  values  and  to  counteract  cognitive  biases  and  illusions,  users  need 
specialized  training  to  better  understand  the  import  of  predictions  and  their  uncertainty 
representations  (Johnson-Laird  et  al.,  1999;  Wickens  and  Hollands,  2000) 

4.3  User  Interface  Designs  for  Agent  Transparency 

4.3.1  Level  1  -  Basic  Information 

The  visualization  of  level  1  information  includes  the  agent’s  purpose  (the  current  goal),  process 
(the  agent’s  intent,  planning  process,  and  the  agent’s  current  progress),  current  performance,  and 
status.  This  may  include  rudimentary  details  about  the  agent’s  planning  process  such  as  what 
routes  it  may  wish  to  take.  A  common  method  of  conveying  basic  planning  information  involves 
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a  top-down  view  of  a  map  with  a  route  overlay.  For  example,  in  FLEX-IT,  which  is  a  prototype 
unmanned  vehicle  control  interface,  potential  paths  are  displayed  using  a  dashed  line  on  top  of  a 
map  of  the  operating  environment  (Calhoun  et  al.,  2012).  These  dashed  lines  become  solid  once 
the  plan  has  been  confirmed  by  the  operator. 

4.3.2  Level  2  -  Rationale 

While  level  linformation  describes  the  what,  level  2  information  conveys  the  why.  The 
visualization  of  level  2  information  is  based  on  a  constraint-driven  reasoning  process.  This  can 
be  conveyed  through  a  representation  of  resource  limitations,  constraints/affordances 
(environmental,  situational,  vehicular,  etc.),  feasibility,  risk,  trade-offs  between  alternatives,  and 
history  of  past  performance. 

The  use  of  lines  overlaid  on  a  map  is  often  beneficial  when  considering  spatial  relationships  and 
restrictions,  such  as  no-fly  zones  and  impassable  terrain  features,  but  is  less  useful  for  abstract 
constraint  information  related  to  the  planning  process  (Kilgore  and  Voshell,  in  press).  For 
example,  for  an  interface  supporting  supervisory  control  of  multiple  unmanned  vehicles,  Kilgore 
and  Voshell  suggested  representing  time-to-target  information  across  several  different  vehicles 
through  icon  size,  where  the  vehicles  that  will  arrive  the  fastest  to  the  target  location  are  the 
largest  relative  to  the  other  icons  in  the  set.  Kilgore  and  Voshell  also  gave  the  example  of  where 
mission  fit  for  the  task  can  be  represented  by  opacity  (see  figure  3).  These  interface  elements 
allow  the  operator  to  select  the  appropriate  assets  and  deploy  them  in  a  more  timely  fashion 
because  the  user  can  quickly  identify  which  unmanned  vehicles  are  appropriately  equipped  and 
can  be  most  quickly  be  summoned  without  having  to  gather  the  various  types  of  information 
from  multiple  sources  and  locations.  An  operator  may  understand  at  a  glance  why  the  agent 
suggested  sending  one  vehicle  over  another  that  was  closer  to  the  target.  Without  the  added 
information  represented  through  icon  opacity,  the  operator  would  not  have  understood  that  the 
obvious  choice — the  closest  vehicle — did  not  have  the  capabilities  needed  for  the  task. 


& 
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Figure  3.  Time  to  target  and  capabilities  can  be  represented  through  icon  size  and  opacity. 
(Reprinted  with  permission  from  Kilgore  and  Voshell  [in  press]). 
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Resource  limitations  affecting  the  agent’s  decisionmaking  process  can  be  visualized  through  an 
icon  array.  This  will  help  an  operator  understand  why  certain  assets  were  chosen  by  the  agent. 

For  fuel  management,  Kilgore  and  Voshell  (in  press)  suggested  using  an  icon  array  that 
represents  the  total  fuel  capabilities  with  different  blocks  and  color-shading  cues  representing 
how  different  aspects  of  the  plan  impact  fuel  levels.  In  their  example,  the  sum  of  all  of  the 
squares  in  the  array  represents  the  total  amount  of  fuel  capacity  while  different  color  sets  of 
blocks  represent  the  amount  of  fuel  required  for  different  plans  in  chronological  order.  Fuel 
reserves  are  represented  by  gray  blocks  and  a  bright  red  line  dictates  the  directive-driven  refuel 
line.  When  new  plans  are  being  considered,  the  fuel  required  may  be  represented  within  the  array 
as  a  new  set  of  highlighted  blocks.  If  the  fuel  consumed  by  the  new  plan  forces  resource  usage 
below  the  refuel  line,  this  will  be  conveyed  to  the  operator  by  turning  the  projected  fuel  blocks 
below  that  bright  red  line.  This  would  allow  operators  to  assess  the  impact  of  the  agent’s 
suggested  plan  in  terms  of  resource  usage. 

4.3.3  Level  3  -  Outcomes 

The  visualization  of  level  3  information  focuses  on  the  projection  to  future  states  (e.g.,  expected 
outcomes,  probabilities  of  success  with  a  confidence  interval)  as  well  as  limitations  (e.g., 
reliability,  likelihood  of  error,  history  of  past  performance).  Level  3  information  is  essential  for 
improving  human-agent  teaming  outcomes.  In  domains  like  autonomous-system  management, 
the  environmental  context  strongly  affects  the  automation’s  accuracy.  For  example,  Neyedli 
et  al.  (2009)  found  that  a  transparent  interface  that  revealed  automation  reliability  resulted  in 
more  appropriate  reliance.  Therefore,  when  appropriate,  information  about  dynamic  system 
confidence  and  the  contextual  nature  of  automation  reliability  should  be  conveyed  to  the  operator 
(Atoyan  et  al.,  2006;  Dong  and  Hayes,  2012;  McGuirl  and  Sarter,  2006;  Seong  and  Bisantz, 
2008).  Cummings  and  Bruni  (2010)  suggested  caution  when  using  visualization  tools  for  mixed- 
initiative  multivariate  planning  tasks,  especially  when  cost  functions  vary  greatly  in  their 
sensitivity  to  changes  in  the  variables  of  the  plan.  Operators’  decisionmaking  may  be  prone  to 
errors  when  dealing  with  more- sensitive  functions  due  to  their  difficulty  in  determining  the  true 
costs  of  different  plan  options.  Proper  uses  of  information  visualization  techniques  can  help 
operators  make  sense  of  information  and  thereby  enhance  their  SA  of  their  mission/tasking 
environments  (Robertson  et  al.,  2009). 

Brust-Renck  et  al.  (2013)  described  multiple  different  ways  risk  can  be  represented  depending  on 
the  context.  Similar  to  EID,  their  method  of  visualization  was  based  on  fuzzy-trace  theory,  in 
which  communicating  the  gist  of  the  message  rather  than  the  exact  numbers  is  beneficial  for 
communicating  risks  to  patients  in  the  medical  domain.  For  example,  shaded  maps  can  be  used 
to  represent  level  of  risk  across  a  geospatial  area;  pairs  of  bar  graphs  can  represent  the  relative 
differences  between  two  magnitudes  and  show  incremental  benefit  (Brust-Renck  et  al.,  2013). 
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Rather  than  traditional,  number-based  representations  of  automation  reliability,  Rovira  et  al.  (in 
press)  explored  the  utility  of  graphical  representations.  The  contextual  information  of  interest 
included  “asset  travel  time,  asset  exposure  to  threats  and  weather,  and  target  coverage”  (Rovira 
et  al.,  p.  3).  Reliability  for  each  contextual  factor  was  influenced  by  aspects  such  as  sensory 
quality  as  affected  by  environmental  conditions  and  age  of  information  (e.g.,  older  information 
degrades  reliability).  Rather  than  providing  a  numerical  value  for  each  regarding  reliability  of 
that  contextual  information,  colors  were  used  where  green  indicated  a  fully  reliable  source, 
yellow  indicated  a  semi-reliable  source,  and  red  indicated  deteriorated  reliability.  Rovira  et  al.’s 
study  required  that  participants  rely  on  both  the  contextual  information  and  information  provided 
by  a  scheduler  display.  They  found  that,  in  high  task  demand  conditions,  those  using  the 
contextual  automated  decision  aid  had  less  of  a  performance  decrement  than  those  who  used  the 
automated  decision  aid  without  it.  They  believed  that  these  results  suggested  that,  in  the  cases  in 
which  perfect  automation  reliability  is  unachievable,  providing  contextual  information  regarding 
the  automation  algorithm’s  reliability  improved  decisionmaking. 

A  unique  approach  to  displaying  multiple  types  of  risk  information  in  one  unique  icon  is  a 
Dynamic  ICON  (DICON)  (deVisser  et  al.,  2014).  De  Visser  et  al.  created  a  decision  support 
system  that  uses  multiple  panels,  colors,  positions,  and  the  various  combinations  of  these 
variables  in  their  DICON  design.  These  panels  represent  various  stages  of  perceived 
trustworthiness  and  actual  trustworthiness  to  inform  the  user  of  actual  calibrated  trust.  The 
DICON  packed  five  levels  of  trust  evidence  into  one  design  to  provide  one  simple  “at  a  glance” 
representation  of  uncertainty  and  trust.  These  five  levels  include  the  current  support  and  known 
risk,  current  best  assumption  and  net  risk,  disagreement  about  risk,  incompleteness  of 
information  about  risk,  and  quality  of  evidence.  Colors  were  used  to  represent  various  levels  of 
trust  from  green  (trust),  black  (miscalibration),  and  red  (resolution).  This  approach  has  two 
potential  drawbacks.  First,  there  are  too  many  potential  combinations  of  variables  to  remember 
and  retrieve  quickly  from  memory,  thus  requiring  a  steep  learning  curve.  Second,  two  or  more 
configurations  may  be  nearly  identical  except  for  one  small  change  in  one  of  the  variables, 
lending  itself  to  confusion  or  an  incorrect  assessment  of  the  DICON. 

This  section  focused  on  how  transparency  can  be  incorporated  into  interfaces  by  considering  the 
different  levels  of  the  SAT  Model.  The  following  sections  present  applied  examples  of 
transparency  display  design  in  human-robot  interaction  contexts.  The  studies  are  part  of  projects 
under  the  Department  of  Defense  Autonomy  Research  Pilot  Initiative. 
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5.  Example:  Autonomous  Squad  Member 


As  technology  continues  to  develop  in  support  of  military  operations,  there  is  a  research  push 
toward  investigation  of  unmanned  systems  and  the  relationship  of  human- agent  teams  (Barnes 
and  Evans,  2010).  There  are  several  efforts  throughout  the  armed  forces,  such  as  the  U.S.  Army’s 
Robotics  Collaborative  Technology  Alliance  (U.S.  Army  Research  Laboratory,  2011)  and  the 
U.S.  Office  of  Naval  Research’s  Maneuver  Thrust  Program  (2013),  that  are  aimed  at  human- 
agent  collaboration.  The  Autonomous  Squad  Member  (ASM)  project  aims  to  increase  agent 
participation  in  dismounted  soldier  reconnaissance  efforts  by  placing  emphasis  on  a  complex 
scenario  where  an  agent  suddenly  has  to  assess  the  situation  (i.e.,  a  firefight  has  broken  out), 
reason,  and  act  accordingly.  The  concept  of  supporting  complex  dynamic  missions  is  consistent 
with  the  Unmanned  Ground  Systems  Roadmap,  which  highlighted  the  need  for  autonomous 
systems  in  combat  logistic  patrols,  support  for  dismounts,  and  reconnaissance  (Robotic  Systems 
Joint  Program  Office,  2012). 

The  research  approach  of  the  ASM  project  begins  with  semantic  understanding,  which  takes  the 
perceptual  data  in  the  environment  and  develops  it  into  an  organized  format  that  the  agent  can 
use  to  comprehend  and  anticipate  the  world  around  it.  Following  this,  the  agent  needs  to  weigh 
the  environmental  and  vehicular  constraints  and  convey  that  to  the  operator.  These  constraints 
are  generated  from  a  goal  reasoning  framework  called  Goal  Driven  Autonomy  (GDA),  which 
takes  the  actions  of  an  agent  needed  to  navigate  and  generates  a  series  of  states  (where  it  is 
currently  and  where  it  wants  to  go),  actions  (the  various  activities  that  can  be  used  to  progress 
toward  the  goal  state),  and  events  that  occur  in  the  environment  and  the  system  needs  to  adjust  to 
(Aha  et  al.,  201 1).  The  events  generated  by  GDA  are  then  passed  to  the  transparency  module, 
where  the  ASM  project  implements  the  tenants  of  the  SAT.  By  examining  each  level 
individually,  informational  content  can  be  created  to  support  the  right  amount  of  information  at 
the  appropriate  time. 

5.1  SAT  Level  1 

In  level  1,  the  ASM  is  trying  to  convey  what  is  going  on  currently  and  what  it  is  trying  to  achieve 
by  accomplishing  its  goals  (i.e.,  task  decomposition).  In  this  state,  the  ASM  is  receiving 
information  from  the  operator,  interpreting  its  meaning,  and  developing  a  series  of  actions  to 
reach  the  end  goal  state,  as  well  as  accomplishing  any  goals  that  occur  while  reaching  the  end 
goal.  The  information  requirements  of  task  decomposition  also  coincide  with  Endsley’s  (1995) 
model  for  stage  1  SA,  as  it  is  the  need  to  understand  a  list  of  proposed  actions  that  the  user 
intends  to  accomplish.  Task  decomposition  content  also  matches  with  Lee’s  (2012)  3Ps:  (1)  for 
purpose,  it  identifies  the  goals  and  objectives  that  need  to  be  accomplished,  (2)  for  process,  the 
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ASM  demonstrates  planning  by  laying  out  the  desired  objectives  and  execution  by  confirming 
that  these  plans  are  feasible  given  its  technical  capabilities,  and  (3)  the  ASM  will  be  assessing  its 
performance  in  understanding  the  operator’s  instructions. 

Another  example  where  SAT  level  1  is  used  is  during  Task  Sequence  Identification  (TSI),  in 
which  the  ASM  provides  the  Soldier  with  a  current  status  of  task  completion  in  terms  of  mission 
goals.  This  can  occur  concurrently  with  other  activities  of  the  squad,  such  as  moving  to  the  next 
reconnaissance  point.  During  the  TSI,  the  operator  assesses  whether  the  ASM  is  acting  in 
accordance  to  intent  for  the  ASM.  If  this  is  true,  the  function  of  the  operator  is  purely 
monitoring.  However,  if  there  are  discrepancies,  they  can  be  identified  quickly  and  efficiently.  It 
will  also  serve  as  a  benchmark  for  the  route  selection/mission  plan  that  the  ASM  developed. 

5.2  SAT  Level  2 

Using  SAT  level  2,  the  user  interface  needs  to  show  the  reasoning  process  of  the  ASM  as  well  as 
explain  the  constraints  that  exist  for  the  environment.  In  essence,  the  ASM  needs  to  reason  about 
its  process  of  reasoning.  This  is  also  known  as  the  ASM’s  metacognition,  the  concept  of  which 
represents  a  person’s  understanding  of  what  thought  processes  are  going  on  within  oneself 
(Flavell,  1976).  In  this  context,  metacognition  refers  to  where  the  ASM  provides  input  as  to  what 
types  of  information  it  is  missing  in  order  for  the  ASM  to  perform  at  its  optimal  level  in  a  given 
state.  For  example,  if  the  ASM  wanted  to  traverse  an  area,  it  would  need  to  first  determine  where 
the  potential  areas  are  to  protect  it  from  enemy  fire.  If,  due  to  the  nature  of  the  terrain  or  the  stream 
of  information  received,  it  did  not  have  that  information,  it  would  need  to  notify  the  operator. 

One  way  to  notify  is  to  incorporate  simple  graphical  designs  called  pictograms,  simple  drawings 
used  to  convey  a  specific  action  or  activity  that  an  end  user  must  take.  Dowse  et  al.  (2010)  found 
that  visuals  that  were  simple,  had  a  clear  focus,  and  reflected  familiar  life  experiences  were 
successful  at  conveying  information.  The  authors  argued  that  pictograms  should  be  used  only  to 
clarify  a  process  or  convey  a  complete  message.  Therefore,  the  development  of  graphical 
elements  to  support  is  being  explored  from  both  a  separate  representations  for  variables  as  well 
as  integrated  visualizations. 

In  the  current  research,  the  ASM  user  interface  expresses  environmental  and  vehicular 
information  through  one  of  three  types  of  information:  needs,  hazards,  and  affordances.  Each  one 
of  these  types  of  information  has  a  different  icon  style  that  is  overlaid  onto  a  Google  Earth  view 
of  potential  navigation  routes.  For  example,  a  blue  icon  with  a  cell  tower  signifies  that  the  ASM 
is  in  need  of  more  information  about  signal  strength.  It  may  be  the  case  that  the  ASM  needs 
continued  connectivity  to  continue  functioning  and  wants  confirmation  from  the  operator  that  the 
signal  will  be  present. 

The  hazard  icon  warns  the  operator  that  if  the  ASM  travels  to  a  particular  area,  there  is  a  possible 
stressor  on  the  ASM  state.  The  ASM  may  get  overheated  if  it  travels  across  an  area  due  to  rough 
terrain.  An  affordance  is  the  opposite  of  a  hazard,  so  that  it  helps  improve  the  performance  of  the 
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ASM.  A  green  eye  icon  indicates  that  there  are  no  restrictions  to  the  line  of  sight  for  the  ASM. 
This  is  important  since  one  of  the  primary  methods  of  information  gathering  for  the  ASM  is 
visual  stimuli. 

5.3  SAT  Level  3 

SAT  level  3  is  leveraged  when  the  ASM  begins  to  help  the  operator  to  understand  what  might 
happen  given  potential  limitations.  The  ASM  assists  the  operator  by  providing  a  visual  display 
demonstrating  the  existing  data  trends  so  far,  an  analysis  of  resource  expenditure  rates,  and  an 
understanding  based  on  what  is  known  about  the  remainder  of  the  mission.  The  ASM  must 
analyze  all  the  data  it  has  collected  up  to  this  point  and  use  that  data  to  make  predictions.  It  can 
also  use  previous  scenario  outcomes  to  increase  its  understanding  if  that  knowledge  is  available. 
By  consistently  increasing  the  size  of  its  reference  model  (i.e.,  learning  from  behavior)  it  can 
produce  more  informed  results. 

Lomas  and  Chevalier  (2012)  developed  a  projection  system  where  a  robotic  entity  was  able  to 
respond  to  questions  asked  to  it  in  terms  of  understandable  reasoning/projections  of  future  state. 
Even  though  the  projections  were  rooted  in  mathematical  models,  it  was  important  to  provide 
users  with  focused  answers  to  facilitate  better  communication  as  to  not  just  what  the  information 
was  but  why  the  entity  believed  it  to  be  so.  It  is  not  sufficient  to  provide  only  information  to  the 
user;  why  this  information  is  valuable  and  how  this  will  change  the  final  outcome  is  also  of 
important.  It  is  this  that  helps  raise  the  SAT  model  beyond  traditional  user-centered  design  (a 
tool  for  someone  to  use)  versus  an  intelligent  agent  to  collaborate  with. 

In  the  development  of  level  3,  there  is  a  focus  on  two  primary  aspects:  projection  and 
uncertainty.  For  the  projection  image  design,  the  work  of  Smallman  and  Cook  (2013)  served  as 
the  foundation.  This  work  incorporates  the  idea  of  a  trendicon,  which  is  an  integrated  variable 
that  represents  more  than  one  quantity  using  basic  perceptual  principles;  for  example,  using 
triangles  to  show  increasing  and  decreasing  quantities  and  color  to  code  overall  system 
state/health.  With  regard  to  uncertainty  visualization,  several  principles  are  developed  from  the 
work  of  MacEachren  et  al.  (2012).  These  include  more  opacity  to  represent  uncertainty, 
increased  distance  from  center  to  represent  uncertainty,  smaller  size  to  represent  uncertainty,  and 
icon  disorganization  to  represent  uncertainty.  Through  the  combination  of  these  two  factors,  the 
goal  is  to  help  the  operator  understand  how  confident  the  agent  is  related  to  information  about  its 
future  state. 
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6.  Example:  Intelligent  Multi-UxV  Planner  With  Adaptive  Collaborative 
/Control  Technologies 


Future  capabilities  of  unmanned  systems  include  reducing  the  ratio  of  operators  to  vehicles. 
Rather  than  manually  controlling  unmanned  vehicles,  this  will  require  a  single  operator  to 
assume  a  supervisory  role  over  multiple  heterogeneous  unmanned  vehicles  (UxVs).  Successful 
implementation  of  such  a  strategy  involves  increased  agility,  workload  management,  and 
optimized  human-automation  teaming.  This  may  be  achieved  through  advanced  automation 
capabilities  involving  cooperative  control  planning  algorithms  (CCAs),  machine  learning  (ML), 
and  advanced  intelligent  agents  (IAs).  The  goal  of  the  Intelligent  Multi-UxV  Planner  with 
Adaptive  Collaborative/Control  Technologies  (IMPACT)  project  is  to  develop  a  prototype 
interface  for  supervisory  control  of  UxVs  that  incorporates  these  systems. 

Given  increasing  amounts  of  supervisory  control  supported  by  IMPACT,  workload  is  inherently 
inflated.  To  address  this,  an  effective  adaptable  interface  through  which  operator-automation 
teams  interact  should  use  the  human  delegation  approach  (Calhoun  et  al.,  2012).  The  human 
delegation  approach  incorporates  natural  language  instructions  (e.g.,  simple,  spoken  commands) 
provided  by  the  supervisor  to  subordinates  based  on  shared  mental  models  (Calhoun  et  al., 

2012).  The  human  delegation  approach  minimizes  workload  as  it  is  an  easy  to  use,  quick,  and 
natural  mode  of  interaction  for  the  operator.  IMPACT  facilitates  the  human  delegation  approach 
in  human-autonomy  interaction  by  incorporating  a  Playbook  system  (Calhoun  et  al.,  2012). 

Playbook  was  developed  by  Miller  et  al.  (2005)  and  encompasses  a  set  of  templates  for  common, 
preplanned  yet  easily  modifiable  maneuvers,  known  as  plays,  from  a  shared  database  that  can  be 
quickly  activated  (“called”)  using  short  commands  that  are  often  verbal  (Miller  et  al.,  2005; 
Calhoun  et  al.,  2012).  In  IMPACT,  the  system  may  suggest  a  play  or  the  operator  may  call  one. 
Once  a  play  has  been  called,  the  CCA,  ML,  and  IA  work  together  to  develop  a  plan.  When 
making  decisions  about  whether  to  use  the  plan  generated  by  the  autonomy,  the  operator  needs  to 
understand  the  decisionmaking  process  executed  by  these  advanced,  interconnected  systems 
(Rovira  et  al.,  in  press).  Increased  transparency  will  help  operators  better  understand  the 
capabilities  of  the  system,  thus  contributing  to  more-informed,  better  decisionmaking,  which 
then  results  in  higher  rates  of  proper  automation  use  based  on  properly  calibrated  trust, 
culminating  in  overall  improved  performance  (Parasuraman  and  Riley,  1997;  Lee  and  See,  2004; 
Beck  et  al.,  2007;  Rovira  et  al.,  in  press).  The  SAT  model  can  be  used  to  provide  guidance  as  to 
what  sort  of  information  should  be  visualized  in  the  IMPACT  interface  to  support  the  operator’s 
understanding  of  how  the  different  autonomous  systems  generated  a  given  plan. 
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6.1  SAT  Level  1 


Recall  that  level  1  information  pertains  to  basic  details  about  the  plan,  such  as  goals,  intent,  and 
planning.  For  IMPACT,  pertinent  level  1  information  includes  the  current  system  status,  the 
assets’  health,  the  current  goal  of  the  play,  the  assets’  role  in  the  current  plan,  and  basic  route 
planning  information.  For  example,  system  status  information  may  include  the  relevant  maps  for 
the  operational  area,  sensor  feed  information,  weather  reports,  alerts,  and  threat  conditions 
(Kilgore  and  Voshell,  in  press).  When  an  automated  system  is  making  decisions  and  is 
transparent,  accurate  system  status  information  must  be  present  so  that  operators  can  check  the 
factors  related  to  the  plan  development  to  the  known  system  state  to  confirm  whether  the 
automation  is  accurate  (Rovira  et  al.,  in  press).  For  example,  to  visualize  threat  condition,  a 
shaded  map,  as  suggested  by  Bmst-Renck  et  al.  (2013),  can  be  used  as  an  overlay  on  top  of  the 
overhead  map  or  in  a  separate  window  solely  for  conveying  threat  information  when  pertinent. 

As  another  example,  vehicle  health  information  should  include  information  pertaining  to 
available  fuel,  current  speed,  altitude,  and  current  time  on  station  (Kilgore  and  Voshell,  in  press). 
One  strategy  for  conveying  this  information  to  the  operator  is  representational  analysis,  where 
different  variables  and  representations  are  identified,  based  on  the  task  hierarchy,  and  placed  into 
context,  i.e.,  displayed  relative  to  data  limits  or  ranges  (Zhang  and  Norman,  1994).  Examples 
include  using  a  tachometer  with  a  red  area  for  maximum  revolutions  per  minute  for  speed  or 
remaining  fuel  out  of  total  fuel  capacity  (Kilgore  and  Voshell,  in  press). 

6.2  SAT  Level  2 

Level  2  information  represents  the  rationale  behind  the  currently  selected  plan.  In  IMPACT,  this 
takes  the  form  of  delivering  information  about  constraints/affordances  within  the  context  of  the 
plan,  specifically  regarding  asset  capabilities,  available  resources,  and  environmental 
constraints/affordances . 

Regarding  asset  capabilities,  information  should  be  present  about  the  vehicle’s  speed,  agility, 
rate  of  fuel  consumption,  range  based  on  fuel,  sensor  capabilities,  autonomous  behavior 
capabilities,  communications  latencies,  and  expected  communications  drop-out  incidence 
(Kilgore  and  Voshell,  in  press).  Rather  than  solely  using  a  drill-down  menu  to  display  specific 
values,  which  increase  operator  workload  as  they  need  to  be  held  in  memory  as  the  menu  will 
disappear  once  the  operator  moves  off  of  it,  capabilities  in  reference  to  the  plan  can  be  visually 
displayed  in  terms  of  suitability  through  relative  icon  size  where  the  larger  asset  icons  indicate 
better  fit  for  achieving  the  current  plan  based  on  asset-specific  capabilities  (Kilgore  and  Voshell, 
in  press). 

In  the  event  of  dropped  communications  between  the  operator  and  the  asset,  the  system  must 
visualize  where  the  communications  were  lost  (a  large  red  X  on  the  vehicle  route,  perhaps  with  a 
small  label  that  states  “comms  lost”  and  lists  the  time),  where  the  asset  was  going  (a  dashed  line 
with  a  similar  red  X  on  the  destination),  where  it  was  programmed  to  go  when  communications 


17 


were  lost  (a  new  dashed  line  with  an  arrow  to  indicate  that  this  is  the  new  route),  and  where  it 
will  reroute  to  once  communications  are  restored  (a  solid  circle  to  represent  the  new  destination 
connected  to  the  new  dashed  line)  (Kilgore  and  Voshell,  in  press). 

Available  resources,  such  as  fuel,  should  be  visualized  within  the  context  of  the  plan.  This  will 
help  an  operator  understand  why  certain  assets  were  chosen  by  the  automated  systems.  For  fuel 
management,  Kilgore  and  Voshell  (in  press)  suggested  using  an  icon  array  that  represents  the 
total  fuel  capabilities,  with  different  blocks  and  color-shading  cues  to  represent  how  different 
aspects  of  the  plan  impact  fuel  levels. 

6.3  SAT  Level  3 

Level  3  information  provides  the  operator  an  understanding  of  the  expected  outcomes.  In 
IMPACT,  this  includes  time  to  complete  objectives,  limitations,  and  probability  of  success  for 
achieving  the  play’s  goals. 

A  timeline  view  with  each  asset  on  its  own  line  with  a  sized-based  representation  of  the  time 
required  for  completing  given  plans  and  subtasks  within  plans  could  be  used  to  represent  the 
amount  of  time  to  complete  objectives.  Such  a  display  would  also  allow  for  a  visual  confirmation 
of  asset  availability  in  the  context  of  all  plans  and  allow  an  operator  to  determine  whether  the 
chosen  plan  fits  the  needs  of  mission  parameters. 

The  limitations  and  reliability  of  the  automated  system  must  be  made  apparent  to  the  operator  so 
that  he/she  can  make  an  informed  judgment  as  whether  to  accept  the  automation’s  suggestion 
(e.g.,  if  the  last  weather  report  is  very  old,  it  may  no  longer  be  accurate  and  render  some  plan 
options  outdated).  As  Rovira  et  al.  (in  press)  suggested,  this  can  be  achieved  by  a  color-coded  list 
of  reliability  for  the  different  factors’  the  system  uses  to  generate  its  plans  (i.e.,  the  factors 
constraining  the  automation’s  decision  discussed  in  level  2).  If  given  a  variety  of  plan  options, 
understanding  the  reliability  of  the  factors  influencing  the  automation’s  decision  helps  the 
operator  project  the  likelihood  of  mission  success.  Confidence  data  can  also  be  tied  to  the  icons 
on  the  map,  such  as  where  wide  margins  around  projected  paths  indicate  execution  uncertainty 
(de  Visser  et  al.,  2014). 


7.  Summary 


In  this  report,  we  have  provided  a  model  and  framework  as  a  foundation  for  transparent 
interfaces  via  our  SAT  model.  We  have  also  provided  examples  of  transparent  interfaces  that  are 
consistent  with  SAT  model  principles  as  well  as  applied  examples.  It  is  suggested  that  the 
research  community  focus  future  efforts  on  empirical  comparisons  of  effective  transparent 
interface  designs  to  compile  guidelines  for  designers  of  future  autonomous  systems.  The  effects 
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of  display  transparency  should  also  be  investigated  in  multiple  domains  to  assess  whether  results 
are  context- specific.  The  additional  information  provided  by  transparent  displays  may  also 
introduce  additional  workload  to  the  user,  so  investigations  as  to  the  effects  of  transparent 
displays  on  workload  should  also  be  performed. 
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12350  RESEARCH  PKWY 
ORLANDO  FL  32826-3276 

1  ARMY  RSCH  LABORATORY  -  HRED 
(PDF)  RDRL  HRM  CU  B  LUT AS -SPENCER 
6501  E  11  MILE  RD  MS  284 
BLDG200A  2ND  FL  RM2104 
WARREN  MI  48397-5000 

1  ARMY  RSCH  LABORATORY  -  HRED 
(PDF)  FIRES  CTR  OF  EXCELLENCE 
FIELD  ELEMENT 
RDRL  HRM  AF  C  HERNANDEZ 
3040  NW  AUSTIN  RD  RM  221 
FORT  SILL  OK  73503-9043 

1  ARMY  RSCH  LABORATORY  -  HRED 
(PDF)  RDRL  HRM  AV  W  CULBERTSON 
91012  STATION  AVE 
FORT  HOOD  TX  76544-5073 

1  ARMY  RSCH  LABORATORY  -  HRED 
(PDF)  HUMAN  RSRCH  AND  ENGRNG 

DIRCTRT  MCOE  FIELD  ELEMENT 
RDRL  HRM  DW  C  CARSTENS 
6450  WAY  ST 
BLDG  2839  RM  310 
FORT  BENNING  GA  31905-5400 

1  ARMY  RSCH  LABORATORY  -  HRED 
(PDF)  RDRL  HRM  DE  A  MARES 

1733  PLEASONTON  RD  BOX  3 
FORT  BLISS  TX  79916-6816 
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COPIES  ORGANIZATION 


8  ARMY  RSCH  LABORATORY  -  HRED 
(PDF)  SIMULATION  AND  TRAINING 
TECHNOLOGY  CENTER 
RDRL  HRT  COL  M  CLARKE 
RDRL  HRT  I  MARTINEZ 
RDRL  HRT  T  R  SOTTILARE 
RDRL  HRT  B  N  FINKELSTEIN 
RDRL  HRT  G  A  RODRIGUEZ 
RDRL  HRT  I  I  HART 
RDRL  HRT  M  C  METEVIER 
RDRL  HRT  S  B  PETTIT 
12423  RESEARCH  PARKWAY 
ORLANDO  FL  32826 

1  ARMY  RSCH  LABORATORY  -  HRED 
(PDF)  HQ  USASOC 

RDRL  HRM  CN  R  SPENCER 
BLDG  E2929  DESERT  STORM  DRIVE 
FORT  BRAGG  NC  28310 

1  ARMY  G1 
(PDF)  DAPE  MR  B  KNAPP 

300  ARMY  PENTAGON  RM  2C489 
WASHINGTON  DC  20310-0300 

13  DIRUSARL 
(PDF)  RDRL  HR 

L  ALLENDER 
P  FRANASZCZUK 
C  COSENZO 
RDRL  HRM 

P  S  A V AGE-KNEPS  HIELD 
RDRL  HRM  AL 
C  PAULILLO 
RFRL  HRM  AR 
J  CHEN 
RDRL  HRM  B 
J  GRYNOVICKI 
RDRL  HRM  C 
L  GARRETT 
RDRL  HRS 
J  LOCKETT 
RDRL  HRS  B 
M  LAFIANDRA 
RDRL  HRS  C 
K  MCDOWELL 
RDRL  HRS  D 
B  AMREIN 
RDRL  HRS  E 
D  HEADLEY 
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